1. Field of the Invention
The present invention generally relates to graphics processing and more specifically to path rendering by covering the path based on a generated stencil buffer.
2. Description of the Related Art
Path rendering is a style of resolution-independent two-dimensional (2D) rendering, often called “vector graphics,” that is the basis for a number of important rendering standards such as PostScript, Java 2D, Apple's Quartz 2D, OpenVG, PDF, TrueType fonts, OpenType fonts, PostScript fonts, Scalable Vector Graphics (SVG) web format, Microsoft's Silverlight and Adobe Flash for interactive web experiences, Open XML Paper Specification (OpenXPS), drawings in Office file formats including PowerPoint, Adobe Illustrator illustrations, and more.
Path rendering is resolution-independent meaning that a scene is described by paths without regard to the pixel resolution of the framebuffer. This is in contrast to the resolution-dependent nature of so-called bitmapped graphics. Whereas bitmapped images exhibit blurred or pixilated appearance when zoomed or otherwise transformed, scenes specified with path rendering can be rendered at different resolutions or otherwise transformed without blurring the boundaries of filled or stroked paths.
Sometimes the term vector graphics is used to mean path rendering, but path rendering is a more specific approach to computer graphics. While vector graphics could be any computer graphics approach that represents objects (typically 2D) in a resolution-independent way, path rendering is a much more specific rendering model with salient features that include path filling, path stroking, dashing, path masking, compositing, and path segments specified as Bèzier curves.
FIG. 1A is a prior art scene composed of a sequence of paths. In path rendering, a 2D picture or scene such as that shown in FIG. 1A is specified as a sequence of paths. Each path is specified by a sequence of path commands and a corresponding set of scalar coordinates. Path rendering is analogous to how an artist draws with pens and brushes. A path is a collection of sub-paths. Each sub-path (also called a trajectory) is a connected sequence of line segments and/or curved segments. Each sub-path may be closed, meaning the sub-path's start and terminal points are the same location so the stroke forms a loop; alternatively, a sub-path can be open, meaning the sub-path's start and terminal points are distinct.
When rendering a particular path, the path may be filled, stroked, or both. As shown in FIG. 1A, the paths constituting the scene are stroked. When a path is both filled and stroked, typically the stroking operation is done immediately subsequent to the filling operation so the stroking outlines the filled region. Artists tend to use stroking and filling together in this way to help highlight or offset the filled region so typically the stroking is done with a different color than the filling.
FIG. 1B is the sequence of paths shown in FIG. 1A with only filling. Filling is the process of coloring or painting the set of pixels “inside” the closed sub-paths of a path. Filling is similar to the way a child would “color in between the lines” of a coloring book. If a sub-path within a path is not closed when such a sub-path is filled, the standard practice is to force the sub-path closed by connecting its end and start points with an implicit line segment, thereby closing the sub-path, and then filling that resulting closed path.
While the meaning of “inside a path” generally matches the intuitive meaning of this phrase, path rendering formalizes this notion with what is called a fill-rule. The intuitive sense of “inside” is sufficient as long as a closed sub-path does not self-intersect itself. However if a sub-path intersects itself or another sub-path or some sub-paths are fully contained within other sub-paths, what it means to be inside or outside the path needs to be better specified.
Stroking is distinct from filling and is more analogous to tracing or outlining each sub-path in a path as if with a pen or marker. Stroking operates on the perimeter or boundary defined by the path whereas filling operates on the path's interior. Unlike filling, there is no requirement for the sub-paths within a path to be closed for stroking. For example, the curve of a letter “S” could be stroked without having to be closed though the curve of the letter “O” could also be stroked.
FIG. 1C is a prior art scene composed of the sequence of paths from FIG. 1A with the stroking from FIG. 1A and the filling from FIG. 1B. FIG. 1C shows how filling and stroking are typically combined in a path rendering scene for a complete the scene. Both stroking and filling are integral to the scene's appearance.
Traditionally, graphics processing units (GPUs) have included features to accelerate 2D bitmapped graphics and three-dimensional (3D) graphics. In today's systems, nearly all path rendering is performed by a central processing unit (CPU) performing scan-line rendering with no acceleration by a GPU. GPUs do not directly render curved primitives so path rendering primitives such as Bèzier segments and partial elliptical arcs must be approximated by lots of tiny triangles when a GPU is used to render the paths. Constructing the required tessellations of a path that is approximated by many short connected line segments can create a substantial CPU burden. The triangles or other polygons resulting from tessellation are then rendered by the GPU. Because GPUs are so fast at rasterizing triangles, tessellating paths into polygons that can then be rendered by GPUs is an obvious approach to GPU-accelerating path rendering.
Tessellation is a fragile, often quite sequential, process that requires global inspection of the entire path. Tessellation depends on dynamic data structures to sort, search, and otherwise juggle the incremental steps involved in generating a tessellation. Path rendering makes this process considerably harder by permitting curved path segments as well as allowing path segments to self-intersect, form high genus topologies, and be unbounded in size.
A general problem with using a GPU to render paths is unacceptably poor antialiasing quality when compared to standard CPU-based methods. The problem is that GPUs rely on point sampling for rasterization of triangular primitives with only 1 to 8 samples (often 4) per pixel. CPU-based scan-line methods typically rely on 16 or more samples per pixel and can accumulate coverage over horizontal spans.
Animating or editing paths is costly because it requires re-tessellating the entire path since the tessellation is resolution dependent, and in general it is very difficult to prove a local edit to a path will not cause a global change in the tessellation of the path. Furthermore, when curved path segments are present and the scaling of the path with respect to pixel space changes appreciably (zooming in say), the curved path segments may need to be re-subdivided and re-tessellation is likely to be necessary.
Additionally, compositing in path rendering systems typically requires that pixels rasterized by a filled or stroked path are updated once-and-only-once per rasterization of the path. This requirement means non-overlapping tessellations are required. So for example, a cross cannot be tessellated as two overlapping rectangles but rather must be rendered by the outline of the cross, introducing additional vertices and primitives. In particular, this means the sub-paths of a path cannot be processed separately without first determining that no two sub-paths overlap. These requirements, combined with the generally fragile and sequential nature of tessellation algorithms make path tessellation particularly expensive. Because of the expense required in generating tessellations, it is very tempting and pragmatic to cache tessellations. Unfortunately such tessellations are much less compact than the original path representations, particularly when curved path segments are involved. Consequently, a greater amount of data must be stored to cache paths after tessellation compared with storing the paths prior to tessellation. Cached tessellations are also ineffective when paths are animated or rendered just once.
Accordingly, what is needed in the art is a robust and efficient system and method for rendering filled and stroked paths without tessellating the paths. Today path rendering systems and methods typically execute on the CPU and are implemented in the context of a scan-line rasterizer; these algorithms do not benefit from the efficient execution model of the GPU. Tessellating filled or stroked paths into triangles for GPU rendering is unattractive for the reasons previously outlined. A technique developed by Charles Loop and Jim Blinn (described in Resolution Independent Curve Rendering using Programmable Graphics Hardware, ACM Transactions on Graphics, Volume 24, Issue 3, July 2005) provide an implicit form for cubic Bèzier curves suitable for efficient evaluation by pixel shaders, but the technique requires the interior of the cubic Bèzier curves to be tessellated into triangles. Other conventional techniques fill paths without tessellation by rendering concave polygons constructed of line segments that are not curved using a stencil buffer. Kokojima et al. (in Resolution Independent Rendering of Deformable Vector Objects using Graphics Hardware, ACM SIGGRAPH 2006 Sketches) describe a tessellation-free approach to filling paths including quadratic Bèzier path segments. However, the approach described by Kokojima et al. is limited to quadratic Bèzier curves and thereby avoids the technical difficulties created by the topological complexity of cubic Bèzier curves. Rueda et.al. (in GPU-based rendering of curved polygons using simplicial coverings. Computers and Graphics, Volume 32, Issue 5, October 2008, pages 581-588.) propose a tessellation-free approach capable of handling cubic Bèzier curves. However, their technique requires Bèzier normalization that results in many times more arithmetic operations per Bèzier curve tested against a point compared with the implicit form of the Bèzier curve developed by Loop and Blinn. Nehab et al. (in Random-access Rendering of General Vector Graphics, ACM Transactions on Graphics, Volume 27, Issue 5, 2007) provide GPU-accelerated system for path rendering. However, their system is limited in the shading and clipping functionality available and demands considerable pre-processing of the scene, making dynamic scene adjustments such as editing difficult. These approaches do not provide a complete method that incorporates the complete set of expected path rendering functionality such as dynamic and editable scenes, arbitrary shading of filled or stroked paths, clipping rendered paths to other arbitrary paths, accurate stroking, and integration with existing 3D rendering. Therefore, the present invention develops a method and system for rendering stroked and filled paths using a programmable GPU that is free of tessellation, suitable for dynamic scenes, capable of clipping rendered paths to arbitrary paths, coloring paths with arbitrary programmable shading, and able to integrate with conventional 3D graphics.
Conventional stroking has been performed by approximating paths into sub-pixel linear segments and then tracing the segments with a circle having a diameter equal to a stroke width. Offset curves are generated at the boundary of the stroked path. These offset curves are typically of much higher degree of complexity compared with the linear segments that are traced to generate the stroked path. Determining whether or not each pixel is inside or outside of a stroked path to generate the stroking is mathematically complex. Identification of the pixels to be stroked is equivalent to identifying pixels that are within half of the stroke width of any point along the path to be stroked. More specifically, the pixels to be stroked are within half of the stroke width measured along a line that is perpendicular to the tangent of the path segment being stroked.
In standard path rendering systems, paths are specified as a sequence of cubic and quadratic (non-rational) Bèzier curve segments, partial elliptical arcs, and line segments. While more mathematically complex path segments representations could be used to specify paths, in practice, existing standards limit themselves to the aforementioned path segment types.
Path filling and stroking use the same underlying path specification. For filling, this means the resulting piece-wise boundaries to be filled may be up to third-order (in the case of cubic Bèzier segments) or rational second-order (in the case of partial elliptical arcs). Filling these curved boundaries of Bèzier curves and arcs is clearly harder than filling the standard polygonal primitives in conventional polygonal 2D or 3D rendering where the boundaries (edges) of the polygonal primitives (usually triangles) are all first-order, being linear segments, and often required to be convex. Filling (and stroking) are also harder than conventional line and convex polygon rasterization because paths are unbounded in their complexity whereas line segments and triangles are defined by just 2 or 3 points respectively. A path may contain just a single path segment or it could contain thousands or more.
The boundaries of stroked paths are actually substantially higher order than the third-order segments. The offset curve of non-rational (second-order) quadratic and (third-order) Bèzier curves are eighth- and tenth-order curves respectively. This high order makes exact determination and evaluation of the resulting offset curves for such Bèzier segments intractable for use in direct rendering. In other words, it is quite unreasonable to try to determine exactly the boundary representation of such offset curves and then simply fill them. For this reason, various techniques have been developed to approximate offset curves with sequences of Bèzier, arc, or line segments. These approximate stroke boundaries may then be filled.
Proper stroking is hard because of the mathematical complexity of the boundary of a path's stroke compared to a path's fill. While approximations to the actual stroke boundary can reduce this complexity, such approximations have associated costs due to inaccuracy and the resulting expansion in the number of primitives that must be both stored and processed to render such approximated strokes. For example, the stroke of a quadratic Bèzier segment can be represented with just the segment's 3 control points (along with the per-path stroke width) whereas an approximation of this stroked boundary with line segments might require dozens or even hundreds of triangles to tessellate approximately the stroked region. Indeed the quality of such tessellations depends on the projection of the curved segment to pixel (or screen) space; this means rendering the same stroked curve at different resolutions would necessitate different tessellations.
Accordingly, what is needed in the art is an improved system and method for rendering paths including filling and stroking of paths.